Software Quality Model Requirements for Software Quality Engineering

Software Quality Engineering is an emerging discipline that is concerned with improving the approach to software quality. It is important that this discipline be firmly rooted in a quality model satisfying its needs. In order to define the needs of this discipline, the meaning of quality is broadly defined by reviewing the literature on the subject. Software Quality Engineering needs a quality model that is usable throughout the software lifecycle and that it embraces all the perspectives of quality. The goal of this paper is to propose a quality model suitable for such a purpose, through the comparative evaluation of existing quality models and their respective support for Software Quality Engineering. Introduction Over the last decade, the general focus of the software industry has shifted from providing ever more functionality to improving what has been coined as the user experience. The user experience refers to characteristics such as ease-of-use, security, stability and reliability. Improvements in such areas lead to an improved quality as perceived by the end users. Some software products, most notably Microsoft's next iteration of their Windows operating system, have been delayed by as much as two years in order to improve their quality. There is no doubt that software quality is becoming an increasingly important subject in software engineering. Traditionally, software requirements have been classified either as functional or nonfunctional with eventual notions of quality hidden in the latter. As the industry focus is shifting from functionality to improving quality, a new category of requirements focused on quality is emerging. In order to define these new quality requirements, quality itself must be defined. A quality model provides the framework towards a definition of quality. Engineers have long recognised that in order for something to find its way in a product, it should be properly defined and specified. Unfortunately, the push towards software quality that can be observed in the industry today is lacking a solid foundation in the form of an agreed upon quality model that can be used not only to evaluate software quality, but also to specify it. Bourque (2000) suggests that the implementation of quality in a software product is an effort that should be formally managed throughout the Software Engineering lifecycle. The implementation of quality should therefore begin with the specification of user quality requirements. Such an approach to the implementation of quality leads to Software Quality Engineering. Suryn (2003) has suggested that this discipline be defined as the application of a continuous, systematic, disciplined, quantifiable approach to the development and maintenance of quality of software products and systems; that is, the application of quality engineering to software. The objective of this paper is to identify the requirements for a software quality model to be used as a foundation to Software Quality Engineering. Definition of Software Quality What exactly constitutes the quality of a product is often the subject of a hot debate. The reason the concept of quality is so controversial is that people fail to agree on what it means. For some it is “[the] degree to which a set of inherent characteristics fulfills requirements” (ISO/IEC 1999b) while for others it can be synonymous with “customer value” (Highsmith, 2002), or even “defect levels” (Highsmith, 2002). A possible explanation as to why any of these definitions fail to garner a consensus is that they generally fail to recognize the different perspectives of quality. Kitchenham and Pfleeger (1996), by reporting the teachings of David Garvin, report on the 5 different perspectives of quality: • The transcendental perspective deals with the metaphysical aspect of quality. In this view of quality, it is “something toward which we strive as an ideal, but may never implement completely.” (Kitchenham & Pfleeger, 1996); • The user perspective is concerned with the appropriateness of the product for a given context of use. Kitchenham and Pfleeger further note that “whereas the transcendental view is ethereal, the user view is more concrete, grounded in the product characteristics that meet user's needs”; • The manufacturing perspective represents quality as conformance to requirements. This aspect of quality is stressed by standards such as ISO 9001, which defines quality as “[the] degree to which a set of inherent characteristics fulfills requirements” (ISO/IEC 1999b). Other models, like the Capability Maturity Model (CMM) state that the quality of a product is directly related to the quality of the engineering process, thus emphasising the need for a manufacturing-like process; • The product perspective implies that quality can be appreciated by measuring the inherent characteristics of the product. Such an approach often leads to a bottom-up approach to software quality: by measuring some attributes of the different components composing a software product, a conclusion can be drawn as to the quality of the end product; • The final perspective of quality is value-based. This perspective recognises that the different perspectives of quality may have a different importance, or value, to various stakeholders. One could argue that in a world where conformance to ISO and IEEE standards is increasingly present in contractual agreements and used as a marketing tool (Adey & Hill, 2000), all the perspectives of quality are subordinate to the manufacturing view. This importance of the manufacturing perspective has increased throughout the years through works like Quality is Free (Crosby, 1979) and the popularity of movements like Six-Sigma (Biehl, 2001). The predominance of the manufacturing view in Software Engineering can be traced back to the 1960s, when the US Department of Defense and IBM gave birth to Software Quality Assurance (Voas, 2003). This has led to the belief that adherence to a development process, as in manufacturing, will lead to a quality product. The corollary to this belief is that process improvement will lead to improved product quality. According to many renowned researchers, this belief is false, or at least flawed. Geoff Dromey states: “The flaw in this approach [that you need a quality process to produce a quality product] is that the emphasis on process usually comes at the expense of constructing, refining, and using adequate product quality models.” (Dromey, 1996) Kitchenham and Pfleeger reinforce this opinion by stating: “There is little evidence that conformance to process standards guarantees good products. In fact, the critics of this view suggest that process standards guarantee only uniformity of output [...]” (Kitchenham & Pfleeger, 1996) Furthermore, data available from Agile (Highsmith, 2002) projects show that high quality is attainable without following a manufacturing-like approach. However, recent studies conducted at Motorola (Eickelman, 2003; Diaz & Sligo, 1997) and Raytheon (Haley, 1996) show that there is indeed a correlation between the maturity level of an organization as measured by the Capability Maturity Model and the quality of the resulting product. These studies provide data on how a higher maturity level (as measured by the CMM) can lead to: • Improved error/defect density (i.e. the error/defect density lowers as maturity improves)